Sound source spatialization system

ABSTRACT

The present invention relates to an enhanced-performance sound source spatialization system used in particular to produce a spatialization system compatible with an integrated modular avionics type system. It comprises a filter database comprising a set of head-related transfer functions specific to the listener, a data presentation processor receiving information from each source and comprising in particular a module for computing the relative positions of the sources in relation to the listener and a module for selecting the head-related transfer functions with a variable resolution suited to the relative position of the source in relation to the listener, a unit for computing said monophonic channels by convoluting each sound source with head-related transfer functions of said database estimated at said source position.

The present invention relates- to an enhanced-performance sound sourcespatialization system used in particular to produce a spatializationsystem compatible with an Integrated Modular Avionics (IMA) type system.

In the field of onboard aeronautical equipment, most thoughts concerningthe cockpit of the future are turned toward the need for a head-upheadset display device, associated with a very large format head-downdisplay. This assembly should improve situation awareness while reducingthe burden of the pilot through a real-time summary display ofinformation deriving from multiple sources (sensors, database).

3D sound falls into the same context as the headset display device byenabling the pilot to obtain spatial situation information (position ofcrew members, threats, etc.) within his own reference frame, via acommunication channel other than visual by a natural method. As ageneral rule, 3D sound enhances the transmitted spatial situationinformation signal, whether the spatial situation is static or dynamic.Its use, besides locating other crew members or threats, can cover otherapplications such as multiple-speaker intelligibility.

In French patent application FR 2 744 871, the applicant described asound source spatialization system producing for each source spatializedmonophonic channels (left/right) designed to be received by a listenerthrough a stereophonic headset, such that the sources are perceived bythe listener as if they originated from a particular point in space,this point possibly being the actual position of the sound source oreven an arbitrary position. The principle of sound spatialization isbased on computing the convolution of the sound source to be spatialized(monophonic signal) with Head-Related Transfer Functions (HRTF) specificto the listener and measured in a prior recording phase. Thus, thesystem described in the abovementioned application comprises inparticular, for each source to be spatialized, a binaural processor withtwo convolution channels, the purpose of which is on the one hand tocompute by interpolation the head-related transfer functions(left/right) at the point at which the sound source will be placed, andon the other hand to create the spatialized signal on two channels fromthe original monophonic signal.

The object of the present invention is to define a spatialization systemoffering enhanced performance so that, in particular, it is suitable forincorporation in an integrated modular avionics (IMA) system whichimposes constraints in particular on the number of processors and theirtype.

For this, the invention proposes a spatialization system in which it isno longer necessary to perform a head-related transfer functioninterpolation computation. It is then possible, to carry out theconvolution operations for creating the spatialized signals, to have nomore than a single computer instead of the n binaural processors neededin the system according to the prior art for spatializing n sources.

More specifically, the invention relates to a spatialization system forat least one sound source creating for each source two spatializedmonophonic channels designed to be received by a listener, comprising:

-   -   a filter database comprising a set of head-related transfer        functions specific to the listener,    -   a data presentation processor receiving the information from        each source and comprising in particular a module for computing        the relative positions of the sources in relation to the        listener,    -   a unit for computing said monophonic channels by convolution of        each sound source with head-related transfer functions of said        database estimated at said source position,        the system being characterized in that said data presentation        processor comprises a head-related transfer function selection        module with a variable resolution suited to the relative        position of the source in relation to the listener.

The use of the databases of transfer functions related to the head ofthe pilot adjusted to the accuracy required for a given information itemto be spatialized (threat, position of a drone, etc.), allied withoptimal use of the spatial information contained in each of thepositions of these databases considerably reduces the number ofoperations to be carried out for spatialization without in any waydegrading performance.

Other advantages and features will become more clearly apparent onreading the description that follows, illustrated by the appendeddrawings which represent:

FIG. 1, a general diagram of a spatialization system according to theinvention;

FIG. 2, a functional diagram of an embodiment of the system according tothe invention;

FIG. 3, the diagram of a computation unit of a spatialization systemaccording to the example in FIG. 2;

FIG. 4, a diagram of implantation of the system according to theinvention in an IMA type modular avionics system.

The invention is described below with reference to an aircraftaudiophonic system, in particular for a combat aircraft, but it isclearly understood that it is not limited to such an application andthat it can be implemented equally in other types of vehicles (land orsea) and in fixed installations. The user of this system is, in thepresent case, the pilot of an aircraft, but there can be a number ofusers thereof simultaneously, particularly in the case of a civiliantransport airplane, devices specific to each user then being provided insufficient numbers.

FIG. 1 is a general diagram of a sound source spatialization systemaccording to the invention, the purpose of which is to enable a listenerto hear sound signals (tones, speech, alarms, etc.) using a stereophonicheadset, such that they are perceived by the listener as if theyoriginated from a particular point in space, this point possibly beingthe actual position of the sound source or even an arbitrary position.For example, the detection of a missile by a counter-measure devicemight generate a sound, the origin of which seems to be the source ofthe attack, enabling the pilot to react more quickly. These sounds(monophonic sound signals) are for example recorded in digital form in a“sound” database. Moreover, the changing position of the sound sourceaccording to the pilot's head movements and the movements of theairplane is taken into account. Thus, an alarm generated at “3 o'clock”should be located at “12 o'clock” if the pilot turns his head 90° to theright.

The system according to the invention mainly comprises a datapresentation processor CPU1 and a computation unit CPU2 generating thespatialized monophonic channels. The data presentation processor CPU1comprises in particular a module 101 for computing the relativepositions of the sources in relation to the listener, in other wordswithin the reference frame of the listener's head. These positions are,for example, computed from information received by a detector 11 sensingthe attitude of the listener's head and by a module 12 for determiningthe position of the source to be restored (this module possiblycomprising an inertial unit, a location device such as a directionfinder, a radar, etc.). The processor CPU1 is linked to a “filter”database 13 comprising a set of head-related transfer functions (HRTF)specific to the listener. The head-related transfer functions are, forexample, acquired in a prior learning phase. They are specific to thelistener's inter-aural delay (the delay with which the sound arrivesbetween his two ears) and the physionomical characteristics of eachlistener. It is these transfer functions that give the listener thesensation of spatialization. The computation unit CPU2 generates thespatialized L and R monophonic channels by convoluting each monophonicsound signal characteristic of the source to be spatialized andcontained in the “sound” database 14 with head-related transferfunctions from said database 13 estimated at the position of the sourcewithin the reference frame of the head.

In the spatialization systems according to the prior art, thecomputation unit comprises as many processors as there are sound sourcesto be spatialized. In practice, in these systems, a spatialinterpolation of the head-related transfer functions is necessary inorder to know the transfer functions at the point at which the sourcewill be placed. This architecture entails multiplying the number ofprocessors in the computation unit, which is inconsistent with a modularspatialization system for incorporation in an integrated modularavionics system.

The spatialization system according to the invention has a specificalgorithmic architecture which in particular enables the number ofprocessors in the computation unit to be reduced. The applicant hasshown that the computation unit CPU2 can then be produced using an EPLD(Embedded Programmable Logic Device) type programmable component. To dothis, the data presentation processor of the system according to theinvention comprises a module 102 for selecting the head-related transferfunctions with a variable resolution suited to the relative position ofthe source in relation to the listener (or position of the source withinthe reference frame of the head). With this selection module, it is nolonger necessary to perform interpolation computations to estimate thetransfer functions at the position where the sound source should belocated. This means that the architecture of the computation unit, anembodiment of which is described below, can be considerably simplified.Moreover, since the selection module selects the resolution of thetransfer functions according to the relative position of the soundsource in relation to the listener, it is possible to work with adatabase 13 of the head-related transfer functions comprising a largenumber of functions distributed evenly throughout the space, bearing inmind that only some of these will be selected to perform the convolutioncomputations. Thus, the applicant worked with a database in which thetransfer functions are collected at 7° intervals in azimuth, from 0 to360°, and at 10° intervals in elevation, from −70° to +90°.

Moreover, the applicant has shown that with the resolution selectionmodule 102 of the system according to the invention, the number ofcoefficients of each head-related transfer function used can be limitedto 40 (compared to 128 or 256 in most systems of the prior art) withoutdegrading the sound spatialization results, which further reduces thecomputation power needed by the spatialization function.

The applicant has therefore demonstrated that the use of the databasesof head-related transfer functions of the pilot adjusted to the accuracyrequired for a given information item to be spatialized, allied withoptimal use of the spatial information contained in each of thepositions of these bases can considerably reduce the number ofoperations to be performed for spatialization without in any waydegrading performance.

The computation unit CPU2 can thus be reduced to an EPLD type component,for example, even when a number of sources have to be spatialized, whichmeans that the dialog protocols between the different binauralprocessors needed to process the spatialization of a number of soundsources in the systems of the prior art can be dispensed with.

This optimization of the computing power in the system according to theinvention also means that other functions which will be described belowcan be introduced.

FIG. 2 is a functional diagram of an embodiment of the system accordingto the invention.

The spatialization system comprises a data presentation processor CPU1receiving the information from each source and a unit CPU2 for computingthe spatialized right and left monophonic channels. The processor CPU1comprises in particular the module 101 for computing the relativeposition of a sound source within the reference frame of the head of thelistener, this module receiving in real time information on the attitudeof the head (position of the listener) and on the position of the sourceto be restored, as was described previously. According to the invention,the module 102 for selecting the resolution of the transfer functionsHRTF contained in the database 13 is used to select, for each source tobe spatialized, according to the relative position of the source, thetransfer functions that will be used to generate the spatialized sounds.In the example of FIG. 2, a sound selection module 103 linked to thesound database 14 is used to select the monophonic signal from thedatabase that will be sent to the computation unit CPU2 to be convolutedwith the appropriate left and right head-related transfer functions.Advantageously, the sound selection module 103 prioritizes between thesound sources to be spatialized. Based on system events and platformmanagement logic choices, concomitant sounds to be spatialized will beselected. All of the information used to define this spatialpresentation priority logic passes over the high speed bus of the IMA.The sound selection module 103 is, for example, linked to aconfiguration and programming module 104 in which customization criteriaspecific to the listener are stored.

The data regarding the choice of head-related transfer functions HRTFand the sounds to be spatialized is sent to the computation unit CPU2via a communication link 15. It is stored temporarily in a filtering anddigital sound memory 201. The part of the memory containing the digitalsounds called “earcons” (name given to sounds used as alarms or alertsand having a highly meaningful value) is, for example, loaded oninitialization. It contains the samples of audio signals previouslydigitized in the sound database 14. At the request of the host CPU1, thespatialization of one or several of these signals will be activated orsuspended. While activation persists, the signal concerned is read in aloop. The convolution computations are performed by a computer 202, forexample an EPLD type component which generates the spatialized sounds ashas already been described.

In the example of FIG. 2, a processor interface 203 forms a memory usedfor the filtering operations. It is made up of buffer registers for thesounds, the HRTF filters, and coefficients used for other functions suchas soft switching and the simulation of atmospheric absorption whichwill be described later.

With the spatialization system according to the invention, two types ofsounds can be spatialized: earcons (or sound alarms) or sounds directlyfrom radios (UHF/VHF) called “live sounds” in FIG. 2.

FIG. 3 is a diagram of a computation unit of a spatialization systemaccording to the example of FIG. 2.

Advantageously, the spatialization system according to the inventioncomprises an input/output audio conditioning module 16 which retrievesat the output the spatialized left and right monophonic channels toformat them before sending them to the listener. Optionally, if “live”communications have to be spatialized, these communications areformatted by the conditioning module so they can be spatialized by thecomputer 202 of the computation unit. By default, a sound originatingfrom a live source will always take priority over the sounds to bespatialized.

The processor interface 203 appears again, forming a short term memoryfor all the parameters used.

The computer 202 is the core of the computation unit. In the example ofFIG. 3, it comprises a source activation and selection module 204,performing the mixing function between the live inputs and the earconsounds.

With the system according to the invention, the computer 202 can performthe computation functions for the n sources to be spatialized. In theexample of FIG. 3, four sound sources can be spatialized.

It comprises a dual spatialization module 205, which receives theappropriate transfer functions and performs the convolution with themonophonic signal to be spatialized. This convolution is performed inthe temporal space using the offset capabilities of the Finite ImpulseResponse (FIR) filters associated with the inter-aural delays.

Advantageously, it comprises a soft switching module 206, linked to acomputation programming register 207 optimizing the choice of transitionparameters according to the speed of movement of the source and of thehead of the listener. The soft switching module provides a transition,with no audible switching noise, on switching from one pair of filtersto the next. This function is implemented by a dual linear weightingramp. It involves double convolution: each sample of each output channelresults from the weighted sum of two samples, each being obtained byconvoluting the input signal with a spatialization filter, an elementfrom the HRTF database. At a given instant, there are therefore in inputmemory two pairs of spatialization filters for each track to beprocessed.

Advantageously, it comprises an atmospheric absorption simulation module208. This function is, for example, provided by a 30-coefficient linearfiltering and single-gain stage, implemented on each channel (left,right) of each track, after spatialization processing. This functionenables the listener to perceive the depth effect needed for his/heroperational decision-making.

Finally, dynamic weighting and summation modules 209 and 210respectively are provided to obtain the weighted sum of the channels ofeach track to provide a single stereophonic signal compatible with theoutput dynamic range. The only constraint associated with thisstereophonic reproduction is associated with the bandwidth needed forsound spatialization (typically 20 kHz).

FIG. 4 diagrammatically represents the hardware architecture of anintegrated modular avionics system 40 of IMA type. It comprises a highspeed bus 41 to which all the functions of the system, including inparticular the sound spatialization system according to the invention42, as described previously, the other man/machine interface functions43 such as, for example, voice control, head-up symbology management,headset display, etc., and a system management board 44 the function ofwhich is to provide the interface with the other aircraft systems, areconnected. The sound spatialization system 42 according to the inventionis connected to the high speed bus via the data presentation processorCPU1. It also comprises the computation unit CPU2, as describedpreviously and for example comprising an EPLD component, compatible withthe technical requirements of the IMA (number and type of operations,memory space, audio sample encoding, digital bit rate).

1. A spatialization system for at least one sound source creating foreach source two spatialized monophonic channels (L, R) designed to bereceived by a listener, comprising: a filter database comprising a setof head-related transfer functions specific to the listener, a datapresentation processor receiving the information from each source andcomprising in particular a module for computing the relative positionsof the sources in relation to the listener, a unit for computing saidmonophonic channels by convolution of each sound source withhead-related transfer functions of said database estimated at saidsource position, wherein said data presentation processor comprises ahead-related transfer function selection module with a variableresolution suited to the relative position of the source in relation tothe listener.
 2. The spatialization system as claimed in claim 1,wherein the head-related transfer functions included in the database arecollected at 7° intervals in azimuth, from 0 to 360°, and at 10°intervals in elevation, from −70° to +90°.
 3. The spatialization systemas claimed in claim 1, wherein the number of coefficients of eachhead-related transfer function is approximately
 40. 4. Thespatialization system as claimed in claim 1, wherein it comprising asound database including in digital form a monophonic sound signalcharacteristic of each source to be spatialized, this sound signal beingdesigned to be convoluted with the selected head-related transferfunctions.
 5. The sound spatialization system as claimed in claim 4,wherein the data presentation processor comprises a sound selectionmodule linked to the sound database prioritizing between the concomitantsound sources to be spatialized.
 6. The sound spatialization system asclaimed in claim 5, wherein the data presentation processor comprises aconfiguration and programming module to which is linked the soundselection module and in which are stored customization criteria specificto the listener.
 7. The spatialization system as claimed in claim 1,wherein it comprises an input/output audio conditioning module whichretrieves at the output the spatialized monophonic channels to formatthem before sending them to the listener.
 8. The spatialization systemas claimed in claim 7, wherein since live communications have to bespatialized, these communications are formatted by the conditioningmodule so they can be spatialized by the computation unit.
 9. The soundspatialization system as claimed in claim 1, wherein the computationunit comprises a processor interface linked with the data presentationunit and a computer for generating spatialized monophonic channels. 10.The sound spatialization system as claimed in claim 9, wherein since thesystem comprises a sound database, the processor interface comprisesbuffer registers for the transfer functions from the filter database andthe sounds from the sound database.
 11. The spatialization system asclaimed in claim 9, wherein the computer is implemented by an EPLD typeprogrammable component.
 12. The spatialization system as claimed inclaim 10, wherein the computer comprises a source activation andselection module, performing the mixing function between livecommunications and the sounds from the sound database.
 13. Thespatialization system as claimed in claim 9, wherein the computercomprises a dual spatialization module which receives the appropriatetransfer functions and performs the convolution with the monophonicsignal to be spatialized.
 14. The spatialization system as claimed inclaim 9, wherein the computer comprises a soft switching moduleimplemented by a dual linear weighting ramp.
 15. The spatializationsystem as claimed in claim 9, wherein the computer comprises anatmospheric absorption simulation module.
 16. The spatialization systemas claimed in claim 9, wherein the computer comprises a dynamic rangeweighting module and a summation module to obtain the weighted sum ofthe channels of each track and provide a single stereophonic signalcompatible with the output dynamic range.
 17. An integrated modularavionics system comprising a high speed bus to which is connected thesound spatialization system as claimed in claim 1 via the datapresentation processor.
 18. The spatialization system as claimed inclaim 11, wherein the computer comprises a source activation andselection module, performing the mixing function between livecommunications and the sounds from the sound database.
 19. Thespatialization system as claimed in claim 10, wherein the computercomprises a dual spatialization module which receives the appropriatetransfer functions and performs the convolution with the monophonicsignal to be spatialized.
 20. The spatialization system as claimed inclaim 10, wherein the computer comprises an atmospheric absorptionsimulation module.